Tidy Data

Artwork by Allison Horst

Same Data, Different Formats

Which data follows a tidy data format?

Tidy Data

Artwork by Allison Horst

Common Types of Data Files

Name, Age
Bob, 49
Joe, 40

Loading External Data

The tidyverse has some cleaned-up versions in the readr and readxl packages:

• read_csv() works like read.csv, with some extra stuff

• read_tsv() is for tab-separated data

• read_table() is for any data with “columns” (white space separating)

• read_delim() is for special “delimiters” separating data

• read_excel() is specifically for dealing with Excel files

Some classroom practice

Let’s:

• Create a new directory called learn_by_doing

• Load the nba_data.xlsx and get acquainted with the data.

• Use here::here() to load the data into R.

APIs in Statistical Computing

Utilizing APIs for Data Access

APIs provide structured, real-time data access, allowing custom queries and often requiring authentication. They are essential in modern data science for accessing updated and specific data sets.

A REST API

:::

Examples of APIs

• Spotify API
  • Analyze music trends and user habits with the spotifyr package, directly interfacing with Spotify’s Web API.
• Finance Data API
  • Access historical stock prices and financial statements using the tidyquant package, integrating with financial markets data.
• Census Data API
  • Retrieve demographics and population statistics with the tidycensus package, simplifying the use of US Census Bureau data.
• Sports Data API
  • Collect sports statistics and performance metrics. For example, use custom-built APIs or data access tools for platforms like Sports Reference.

:::

Grammar of Graphics: graphic forms from the ground up

Think of a data visualization or graph as a mapping

• from variables in the data set, (or statistics computed from the data)
• to visual attributes (or “aesthetics”) of marks (or “geometric elements”) on the page/screen

Grammar of Graphics: components

GoG components, as specified in R’s ggplot2

• data
• aes : aesthetic mappings (position, length, color, symbol, …)
• geom : geometric element (point, line, bar, …)
• stat : statistical variable transformation (identity, count, linear model, quantile, …)
• scale : scale transformation (log scale, color mapping, axes tick breaks, …)
• coord : Cartesian, polar, map projection, …
• facet : divide into subplots / small multiples using a categorical variable

Of course, we can also control axes, legends, titles … (guides)

How to build a graph

How to build a graph

ggplot(data = mpg)

ggplot(data = mpg, 
       mapping = aes(x = class, y = hwy)
       )

ggplot(data = mpg, 
       mapping = aes(x = class, y = hwy)
       ) +
  geom_jitter()

ggplot(data = mpg, 
       mapping = aes(x = class, y = hwy)
       ) +
  geom_jitter() +
  geom_boxplot()

Aesthetics

In ggplot2, we map variables from the data set to aesthetics on the chart

ggplot(data = txhousing, 
       mapping = aes(
         x = date, 
         y = median, 
         color = city)) + 
  geom_point() + 
  geom_smooth(method = "loess") + 
  xlab("Date") + 
  ylab("Median Home Price") + 
  ggtitle("Texas Housing Prices")

ggplot(data = housingsub, 
       mapping = aes(x = date, 
                     y = median)
       ) +
  geom_point()

ggplot(data = housingsub) +
  geom_point(mapping = aes(x = date, 
                           y = median)
             )

ggplot(data = housingsub) +
  geom_point(mapping = aes(x = date, 
                           y = median,
                           color = city)
             )

ggplot(data = housingsub) +
  geom_point(mapping = aes(x = date, 
                           y = median), 
             color = "blue"
               )

Geometric objects

In ggplot2, we use a geom function to represent data points, and use the geom’s aesthetic properties to represent variables.

ggplot(data = mpg,
       aes(x = cty,
           y = hwy,
           color = class)
       ) +
  geom_point() +
  labs(xlab = "City (mpg)", y = "Highway (mpg)")

ggplot(data = mpg,
       aes(x = cty,
           y = hwy,
           color = class)
       ) +
  geom_text(aes(label = class)) +
  labs(xlab = "City (mpg)", y = "Highway (mpg)")

Alternative method of building layers: Stats

A stat builds a new variable to plot (e.g., count and proportion)

Faceting

A way to extract subsets of data and place them side-by-side in graphics

ggplot(data = mpg, aes(x = cty, y = hwy, color = class)) + 
  geom_point() +
  facet_grid(~class)

Position Adjustements

Position adjustments determine how to arrange geoms that would otherwise occupy the same space

• position = 'dodge': Arrange elements side by side
• position = 'fill': Stack elements on top of one another, normalize height
• position = 'stack': Stack elements on top of one another
• position = 'jitter": Add random noise to X & Y position of each element to avoid overplotting (see geom_jitter())

ggplot(mpg, aes(fl, fill = drv)) + 
  geom_bar(position = "")`

Plot Customizations

ggplot(data = mpg) + 
  geom_jitter(mapping = aes(x = displ, y = hwy, color = cyl)) + 
  labs(x = "Engine Displacement (liters)", 
       y = "Highway MPG", 
       color = "Number of \nCylinders",
       title = "Car Efficiency")

ggplot(data = mpg) + 
  geom_jitter(mapping = aes(x = displ, y = hwy, color = cyl)) + 
  labs(xlab = "Engine Displacement (liters)", 
       ylab = "Highway MPG", 
       color = "Number of \nCylinders",
       title = "Car Efficiency") +
  theme_bw() +
  theme(legend.position = "bottom")

ggplot(data = mpg) + 
  geom_jitter(mapping = aes(x = displ, y = hwy, color = cyl)) + 
  labs(x     = "Engine Displacement (liters)",
       color = "Number of \nCylinders",
       title = "Car Efficiency") +
  scale_y_continuous("Highway MPG", 
                     limits = c(0,50),
                     breaks = seq(0,50,5)
                     )

ggplot(data = mpg) + 
  geom_jitter(mapping = aes(x = displ, y = hwy, color = cyl)) + 
  labs(x    = "Engine Displacement (liters)",
       y    = "Highway MPG",
       color = "Number of \nCylinders",
       title = "Car Efficiency") +
  scale_color_gradient(low = "white", high = "green4")

Formatting your Plot Code

ggplot(data = mpg, mapping = aes(x = cty, y = hwy, color = class)) + geom_point() + theme_bw() + labs(x = "City (mpg)", y = "Highway (mpg)")

ggplot(data = mpg, 
       mapping = aes(x = cty, 
                     y = hwy, 
                     color = class)
       ) + 
  geom_point() + 
  theme_bw() + 
  labs(x = "City (mpg)", 
       y = "Highway (mpg)"
       )

ggplot(data = mpg, 
       mapping = aes(x = cty, y = hwy, color = class)
       ) + 
  geom_point() + 
  theme_bw() + 
  labs(x = "City (mpg)", y = "Highway (mpg)")

PA 2: Using Data Visualization to Find the Penguins

Artwork by Allison Horst

To do…

• PA 2: Using Data Visualization to Find the Penguins
  • Due TOMORROW, Wednesday (1/18) at 8:00AM
• Bonus Challenge: Ugly Graphics of Penguins (+2)
  • Due TOMORROW, Wednesday (1/18) at 10:10AM

Further

Today we will…

• Review PA 2: Using Data Visualization to Find the Penguins
• Ugly Graphics of Penguins
• Mini lecture on text material
  • What makes a good graphic?
• Lab 2: Exploring Rodents with ggplot2
• Challenge 2: Spicing things up with ggplot2

Why are some plots easier to read than others?

What makes bad figures bad?

Good Graphics

Graphics consist of:

• Structure: boxplot, scatterplot, etc.

• Aesthetics: features such as color, shape, and size that map other characteristics to structural features

Both the structure and aesthetics should help viewers interpret the information.

Gestalt Principles

What sorts of relationships are inferred, and under what circumstances?

Gestalt Principles

Implications for practice

• Know how we perceive groups
• Know that we perceive some groups before others
• Design to facilitate and emphasize the most important comparisons

Pre-attentive Features

Pre-attentive Features

Pre-attentive Features

Pre-Attentive Features are things that “jump out” in less than 250 ms

• Color, form, movement, spatial localization

There is a hierarchy of features

• Color is stronger than shape
• Combinations of pre-attentive features are usually not pre-attentive due to interference

Pre-attentive Features: Double Encoding

Pre-attentive Features: Double Encoding

Color: Implications and Guidelines

• Do not use rainbow color gradient schemes.
• Avoid any scheme that uses green-yellow-red signaling if you have a target audience that may include colorblind people.
• To “colorblind-proof” a graphic, you can use a couple of strategies:
  • double encoding - where you use color, use another aesthetic (line type, shape)
  • If you can print your chart out in black and white and still read it, it will be safe for colorblind users. This is the only foolproof way to do it!
  • If you are using a color gradient, use a monochromatic color scheme where possible.
  • {If you have a bidirectional scale (e.g. showing positive and negative values), the safest scheme to use is purple - white - orange. In any color scale that is multi-hue, it is important to transition through white, instead of from one color to another directly.}
• {Be conscious of what certain colors “mean”}

Gradients

Color in ggplot2

There are packages available for use that have color scheme options.

Some Examples:

• Rcolorbrewer
• ggsci
• viridis
• wes anderson

There are packages such as RColorBrewer and dichromat that have color palettes which are aesthetically pleasing, and, in many cases, colorblind friendly.

You can also take a look at other ways to find nice color palettes.

Week 2 Assignments

Lab 2: Exploring Rodents with ggplot2

To do…

• Lab 2: Exploring Rodents with ggplot2
  • due Friday, 1/20 at 11:59pm
• Read Chapter 3: Data Cleaning and Manipulation
  • Concept Check 3.1 + 3.2 due Monday (1/23) at 8am